Uni Duisburg-Essen Logo HECF Logo

ENTSO-E API Python Package

Streamlining Access to European Electricity Market Data

Jonathan Berrisch

University of Duisburg-Essen, House of Energy, Climate, and Finance

December 9, 2025

Who we are

The Chair of Data Science in Energy and Environment

Prof. Dr. Florian Ziel

Focus

Methodological

  • Online Learning
  • Regularization
  • Multivariate and Probabilistic and Forecasting

Applied:

  • Forecasting in Energy Markets
  • Electricity Price and Load Forecasting
  • Water:
    • Demand
    • Wastewater Supply

ENTSO-E

Introduction


  • European Network of Transmission System Operators for Electricity
  • Formation: 19th December 2008
  • Legal mandate from European Union’s Third Energy Package
  • Representing 40 electricity transmission system operators (TSOs) from 36 countries across Europe
  • Promotes closer cooperation across Europe’s TSOs

Transparency Platform (TP)

  • Central Information Transparency Platform
  • Provides free access to fundamental data and information on pan-European wholesale energy
    • generation
    • transmission
    • consumption
      • Access via: SFTP, REST API

Discontinuation of the ENTSOE SFTP Server

ENTSOE announced to shutdown SFTP server end of September 2025​

Later postponed until “mid November”​

We queried that server hourly to obtain​:

  • Prices
  • Generation
  • Load
  • Unavailabilities
  • etc.​

Remaining options​

  • Obtain CSV Extracts via RESTful API​

  • Obtain data as XML via RESTful API

Accessing the RESTful API

  • Dealing with the API directly is tedious​
    • XML Files need to be parsed​
    • API has many quirks that need to be handled​
  • Right screenshot shows search results for “Entsoe” on PyPI​
    • Most Packages​
      • deprecated / unmaintained​
      • Incomplete (code + documentation)​
    • entsoe-py​
      • Very popular, “State of the Art”​
      • Documentation is a single readme file​
      • Results sometimes inconsistent​
      • Rewrite vs. backwards compatibility​
  • Rewrite from Scratch => entsoe-apy

ENTSOE-APY Python Package

Overview

In a nutshell:​

  • Easy access to all endpoints​
  • Extensive documentation: entsoe-apy.berrisch.biz/​
  • Rigorous consistency with ENTSOE’s documentation​

Technical Overview​

  • We parse XML responses (using ENTSOE’s XSD schema) into pydantic python classes​
  • Converting data to JSON / pandas dataframes is straightforward​
  • Almost no postprocessing:​
    • Users get the entire response (+Responsibility)​
  • Goals:​
    • Consistency​ & Maintainability & Stability​

Structure

Main Classes

Used to build a query.

We mirror ENTSOE’s documentation:

entsoe.Market.ActualTotalLoad
entsoe.Market.DayAheadTotalLoadForecast
entsoe.Market.WeekAheadTotalLoadForecast
entsoe.Market.MonthAheadTotalLoadForecast
entsoe.Market.YearAheadTotalLoadForecast
entsoe.Market.YearAheadForecastMargin
entsoe.Load.<...>
entsoe.Generation.<...>
entsoe.Transmission.<...>
entsoe.Outages.<...>
entsoe.Balancing.<...>
entsoe.Master Data.<...>
entsoe.OMI.<...>

Each class holds .params dictionary

Each class provides .query method.

Utilities

entsoe.utils.extract_records
entsoe.utils.add_timestamps
entsoe.utils.calculate_timestamp
entsoe.utils.mappings = {
    "10YBE----------2": {"BE": ["BZN", "CTA", "CTY", "LFA", "LFB", "MBA", "SCA"]},
    "10YIE-1001A00010": {"IE": ["CTA", "CTY", "SCA"], "SEM(EirGrid)": ["MBA"]},
    "10Y1001A1001A59C": {
        "IE(SEM)": ["BZN", "MBA", "SCA"],
        "IE-NIE": ["LFB"],
        "Ireland": ["SNA"],
    },
}
entsoe.codes.<...> # Enums for translation

Configuration

entsoe.config.set_config
entsoe.config.get_config

Internal

Internal modules: entsoe.xml_models, entsoe.query

Example


The package will read API key from ENTSOE_API environment variable

from pandas import DataFrame
from entsoe.Market import EnergyPrices
from entsoe.utils import extract_records, add_timestamps

# from entsoe.config import set_config
# set_config(security_token=None, endpoint_url=None, timeout=5, retries=5,
#            retry_delay=lambda attempt: 2**attempt, max_workers=4, log_level="SUCCESS"
#           )

result = EnergyPrices(
    in_domain="10Y1001A1001A82H",  # Germany, DE-LU Biddingzone
    out_domain="10Y1001A1001A82H",
    period_start=202012312300,
    period_end=202101022300,
).query_api()

records = extract_records(result)  # Serialize <=> Convert from pydantic class to dict
records = add_timestamps(records)
df = DataFrame(records)

Features

  • Automatically splits up requests
    • If requested range exceeds endpoint-specific limit
  • Parallelism using ThreadPoolExecutor
  • Thread-safe rate-limiting to stay within API limits
  • Sophisticated retry mechanism
    • Retry on
      • service unavailable (502)
      • requester banned (429)
      • Acknowledgement with Unexpected Error
      • Errors from httpx
    • Retry after \(2^n\) seconds where \(n\) is the attempt
  • Sophisticated logging via loguru
    • Configurable via entsoe.config.set_config(loglevel = "DEBUG")
  • Takes care of pagination (the offset parameter)
  • Handles .zip responses

Execution Flow

@split_date_range
@pagination
def query_api(params):
    results = query_and_parse(params)
    return results


@retry
def query_and_parse(params):
    responses = fetch_responses(params)
    xml_models = parse_response(responses)
    return xml_models


@unzip
def fetch_responses(params) -> list[Response]:
    response = query_core(params)
    return [response]


@handle_acknowledgement
def parse_response(response):
    _, matching_class = extract_namespace_and_find_classes(response)
    xml_model = XmlParser().from_string(response.text, matching_class)
    return xml_model

Retry on 502 Responses (API Unavailable)

Decorated Query Core

@check_if_banned
@check_service_unavailable
@rate_limit(max_calls=380, period=60)
def query_core(params):
    params_with_token = {**params, "securityToken": get_config().security_token}
    response = get(config.endpoint_url, params=params_with_token, timeout=config.timeout)
    return response

Unavailability Decorator

def check_service_unavailable(func):
    @wraps(func)
    def service_unavailable_wrapper(*args, **kwargs):
        response = func(*args, **kwargs)
        if response.status_code == 503:
            # @retry decorator will check for this error
            raise ServiceUnavailableError("ENTSO-E API is unavailable (HTTP 503).")
        return response

    return service_unavailable_wrapper

Decorators create a hidden layer to handle exceptions, without cluttering the core logic.

Wrap-Up

  • Streamlined fast access to ENTSOE’s API
  • Extensive documentation
  • Easy to use
  • Consistent with the API
  • Available on GitHub, pypi, and nixpkgs
  • Contributions are welcome

Roadmap:

  • 100% test coverage

berrisch.biz/slides/25_12_RenEU